United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 

Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 223 1 3- 1 450 
www.uspto.gov 



APPLICATION NO. 


FILING DATE 


FIRST NAMED INVENTOR 


ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


09/805,748 


03/13/2001 


Srinivas Gutta 


US 010064 


2712 



24737 



7590 



1 1/03/2004 

PHILIPS INTELLECTUAL PROPERTY & STANDARDS 
P.O. BOX 3001 

BRIARCLIFF MANOR, NY 10510 



EXAMINER 



FLETCHER, JAMES A 



ART UNIT 



PAPER NUMBER 



2616 

DATE MAILED: 1 1/03/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



kjtticg Mciion ou ni friary 


Application No. 

09/805,748 


Appiicant(s) y V * 
GUTTAETAL. 


Examiner 

James A. Fletcher 


Art Unit 

2616 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address ~ 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )IEI Responsive to communication(s) filed on 13 March 2001 . 
2a)D This action is FINAL. 2b)IEl This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) IEI Claim(s) 1-60 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [X] Claim(s) 1-60 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 06 August 2001 is/are: a)S accepted or b)\3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d), 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-1 52. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)Q Some * c)D None of: 

1 0 Certified copies of the priority documents have been received. 

20 Certified copies of the priority documents have been received in Application No. . 

3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mail Date. . 

3) ^ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 ) D Notice of Informal Patent Application (PTO-152) 

Paper No(s)/Mail Date 3/13/1 and 1/23/3 . 6) □ Other: . 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 1-04) 



Office Action Summary 



Part of Paper No./Mail Date 041001 



Application/Control Number: 09/805,748 Page 2 

Art Unit: 2616 

DETAILED ACTION 

1 . Please include the new Art Unit 261 6 in the caption or heading of any written or 
facsimile communication submitted after this Office Action because the examiner, who 
was assigned to Art Unit 2615, will be assigned to new Art Unit 2616. Your cooperation 
in this matter will assist in the timely processing of the submission and is appreciated by 
the Office. 

Claim Rejections - 35 USC § 103 

2. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

3. Claims 1-6, 8-36, and 38-0 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Yeo et al (6,219,837), in further view of Dimitrova et al (6,137,544). 

Regarding claims 1 and 31, Yeo et al disclose a system and method for 
processing video source frames comprising: 

• a video frame extraction algorithm that dynamically and non-contiguously 
extracts key frames from the video source frames during execution of the 
video source frames (Col 4, lines 67-68 "A candidate summary frame is taken 
from the video signal"); 

• a processor that executes the video source frames and executes the video 
source frame extraction algorithm (Col 4, lines 15-16 "the processing of video 
and embedding of summary is done on a server"); 
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• a video input device that receives the video source frames from a video 
source, wherein the video input device is coupled to the processor (Fig 4, 
showing "Video Signal" 403 directly coupled to "Processing of Video Signal" 
404"); 

• wherein a first memory of the memory structure stores the extracted key 
frames (Col 5, lines 1-2 "A fixed memory buffer in a computing device"); 

• Yeo et al disclose a memory of the processor (Col 5, lines 1 -2 "A fixed 
memory buffer in a computing device"), but do not specifically disclose an 
algorithm stored in that memory. 

Dimitrova et al teach an algorithm stored in the memory structure 
coupled to the processor. (Fig. 2A, item 234 "Frame Memory" is shown 
coupled to item 230 "Significant Scene Processor") wherein the memory 
structure stores the video frame extraction algorithm (The processor is shown 
to be an Intel Pentium chip in Col 4, lines 29-30, and such chips are known to 
require programs stored in memory to function, the examiner determines that 
the limitation is met by the Dimitrova et al reference). 

Algorithms for processing various kinds of data by processors are well 
known, widely used and commercially available, and provide an inexpensive 
way to provide a functionality of key frame extraction to a processor. 

Therefore, it would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Yeo et al to specify a key frame 
extraction algorithm stored in the memory coupled to the processor. 
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• Yeo et al disclose a way to terminate key frame viewing (Col 3, lines 44-46 
"the viewer 10 may select to delete the summary frames"), but do not 
specifically disclose terminating extraction of key frames. 

Dimitrova et al teach a terminating mechanism that terminates 
extraction of the key frames prior to completion of execution of the video 
source frames (Col 3, lines 39-42 "if the tape, or file, is not completely 
recorded on at one time, a partially created video index could be saved on the 
tape, file, etc."). 

As suggested by Yeo et al and taught by Dimitrova et al, when the key 
frames are no longer of interest to the viewer, their extraction and display are 
no longer necessary, and stopping their extraction and display would free 
processor time for other tasks that may have more value to the viewer. 

Therefore, it would have been obvious to one of ordinary skill in the art 
at the time of the invention to modify Yeo et al to terminate key frame 
extraction before all frames have been executed. 
Regarding claims 2, 4, 5, 32, 34, and 35, Yeo et al suggest a memory for 
source frames (Col 5, lines 1-2 "A fixed memory buffer in a computing device can be 
maintained to store the selected frames", but does not specifically disclose either a 
temporary or permanent memory, or what form a permanent memory might take. 

Dimitrova et al teach a system and method for processing video source frames 
wherein the first memory includes a temporary or permanent memory, and includes 
forms of those memories (Col 4, lines 35-38 "an index memory, such as, for example, a 
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hard disk, file, tape, DVD, or other storage medium" The examiner notes that such 
media are well known to those of ordinary skill in the art to be useful for both temporary 
and permanent storage of data.). 

As suggested by Yeo et al and taught by Dimitrova et al, both permanent and 
temporary memory devices and methods are well known, commercially available, and 
widely used for storage of image data, and allow images to be stored quickly and 
cheaply. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to specify temporary and permanent memory 
devices. 

Regarding claims 3 and 33, Yeo et al suggest but do not specifically disclose a 
system and method for processing video source frames wherein the temporary memory 
includes a random access memory (RAM) (Col 5, lines 1-2 "A fixed memory buffer in a 
computing device can be maintained to store the selected frames"). 

The examiner takes official notice that a random access memory is a well known, 
widely used and commercially available means of temporary storage of data, and would 
provide an inexpensive and easily understood means of storing such data. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Martino et al to specify a random access memory for 
temporary storage of data. 

Regarding claims 6 and 36, Yeo et al suggest indicating a key frame at the end 
of extraction of key frames (Col 5, lines 4-5 "In a live-broadcast, summary frames of 
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past content can be selected." The most recent key frame is indicated by its position on 
the screen as shown in Figs 8A, 8B, and 8C). 

Dimitrova et al teach a system and method for processing video source frames 
comprising a recording mechanism that records in the first memory an indication of a 
video source frame being executed when the extraction of key frames is terminated (Fig 
1 illustrates a flow chart that indicates the recording of keyframe data to a tape memory 
as a final step in the acquisition of keyframe data). 

As suggested by Yeo et al and taught by Dimitrova et al, indicating the final 
frame in a set of extracted key frames would provide the user with information regarding 
how recently the key frame review data can be accesses, further enhancing the use of 
the key frame extraction and display system to him or her. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to include an indication of the last extracted key 
frame. 

Regarding claims 8 and 38, Yeo et al disclose a way to terminate key frame 
viewing (Col 3, lines 44-46 "the viewer 10 may select to delete the summary frames"), 
but do not specifically disclose terminating extraction of key frames. 

Dimitrova et al disclose a system and method for processing video source frames 
wherein the terminating is triggered by action of a user-controlled device (Col 13, lines 
44-46 "An additional feature would allow a user to stop the playing of a video tape at 
any point and access the visual index for that video tape"). 
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As taught by Dimitrova et al, when the key frames are no longer of interest to the 
viewer, their extraction and display are no longer necessary, and stopping their 
extraction and display would free processor time for other tasks that may have more 
value to the viewer. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to terminate key frame extraction before all 
frames have been executed. 

Regarding claims 9 and 39, Yeo et al disclose a system and method for 
processing video source frames wherein the user-controlled device includes a user 
input that is coupled to the processor (Col 3, lines 44-46 "the viewer 10 may select to 
delete the summary frames"). 

Regarding claims 10 and 40, Yeo et al disclose a system and method for 
terminating the viewing of keyframes at a time when a predetermined condition has 
occurred (Col 3, lines 56-48 "the summary screens may automatically disappear from 
the screen if the channel has not been changed for some time"), but do not specifically 
state that the extraction terminates at the occurrence of a predetermined condition. 

Dimitrova et al teach terminating the processing of video source frames when a 
predetermined condition has occurred (Col 3, lines 40-43 "if the tape, or file, is not 
completely recorded on at one time, a partially created video index could be saved on 
the tape, file, etc. or could be saved in a tape memory for later additions"). 

As suggested by Yeo et al and taught by Dimitrova et al, when the key frames 
are no longer of interest to the viewer, their extraction and display are no longer 
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necessary, and stopping their extraction and display would free processor time for other 
tasks that may have more value to the viewer. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to terminate key frame extraction before all 
frames have been executed. 

Regarding claims 11-12 and 41-42, Yeo et al disclose a system and method for 
terminating the viewing of key frames at after a predetermined time has passed (Col 3, 
lines 56-48 "the summary screens may automatically disappear from the screen if the 
channel has not been changed for some time"), but do not specifically state if that 
predetermined time is a passage of clock time, a given number of frames being 
executed, or a given fraction of the program being observed. 

The examiner takes official notice that a predetermined fraction of the video 
source frames, and a predetermined number of video source frames are both well 
known and widely used techniques of determining the passage of time in an 
audio/visual program. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to indicate the predetermined condition by a 
percentage of the incoming program or a count of incoming frames. 

Regarding claims 13 and 43, Yeo et al disclose a system and method for 
indicating the predetermined condition as the elapsing of a predetermined time duration 
from initiation of executing the video source frames (Col 3, lines 56-48 "the summary 
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screens may automatically disappear from the screen if the channel has not been 
changed for some time"). 

Regarding claims 14 and 44, Yeo et al disclose a system and method for 
processing video source frames comprising an output display through which a user may 
review the extracted key frames (Col 3, lines 25-26 "summary frames... appear in a 
small window as shown on the left side of the screen 18"), wherein the output display is 
coupled to the processor (Fig 4 shows the displays 412 and 410 coupled to the 
processor through the communication channel 402). 

Regarding claims 15 and 45, Dimitrova et al disclose a system and method for 
processing video source frames wherein the output display includes a television screen 
or a computer monitor (Fig 4 shows the output display as being available on either a 
television screen 41 2 or a computer monitor 41 0). 

Regarding claims 16-18 and 46-48, Yeo et al disclose a system and method for 
processing video source frames wherein the system permits review of the key frames 
prior to, when, or after completion of execution of the video source frames, or before the 
terminating mechanism terminates extracting the keyframes (Col 3, lines 25-26 "These 
summary frames... appear in a small window as shown"). 

Regarding claims 19 and 49, Yeo et al disclose a system and method for 
processing video source frames wherein the system permits review of the key frames 
upon or after completion of execution of the video source frames (Col 3, lines 24-27 
"summary frames are selected from the video prior to broadcast. These summary 
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frames are embedded in the broadcast and appear in a small window as shown on the 
left side of the screen 18"). 

Regarding claims 20, 21, 50, and 51, Yeo et al disclose a system and method 
for processing video source frames comprising an erasing mechanism that erases the 
key frames from the first memory at or after completion of review of the key frames by 
the user, and that action is triggered by action of the user (Col 3, lines 44-46 "the viewer 
10 may select to delete the summary frames from the screen if they wish"). 

Regarding claims 22 and 52, Yeo et al disclose a system and method for 
processing video source frames comprising a user input device for user manipulation 
(Col 3, lines 37-42 "A short video segment may be associated with each shot such that 
when the shot is selected... the past video segment is played in a larger window on the 
screen"). 

Regarding claims 23 and 53, Yeo et al disclose a system and method for 
processing video source frames wherein the erasing mechanism is triggered when a 
predetermined condition has occurred (Col 6, lines 6-9 "At block 33, S sj is removed from 
S is S S j is older than some time t. This ensures that summary frames displayed on the 
screen are not too old to be relevant"). 

Regarding claims 24 and 54, Yeo et al disclose a system and method for 
processing video source frames wherein the erasing mechanism is triggered when a 
predetermined condition has occurred, if that condition includes completion of execution 
of the video source frames (Col 2, lines 17-18 "the summary frames may also be 
embedded in prerecorded media" and Col 6, lines 6-9 "At block 33, S sj is removed from 
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S is S S j is older than some time t. This ensures that summary frames displayed on the 
screen are not too old to be relevant"). 

Regarding claims 25 and 55, Yeo et al disclose a system and method for 
processing video source frames wherein the predetermined condition includes an 
elapse of a predetermined amount of time following the review of the key frames (Col 3, 
lines 46-48 "the summary screens may automatically disappear from the screen if the 
channel has not been changed for some time"). 

Regarding claims 26 and 56, Yeo et al disclose a system and method for 
processing video source frames comprising a second memory of the memory structure 
and a transferring mechanism, wherein the transferring mechanism transfers the key 
frames from the first memory to the second memory (Fig. 4 illustrates the embedding of 
frames into the video signal from the server, and that signal being transferred to users' 
computers or set-top boxes), and wherein the second memory includes a removable 
memory (Col 2, lines 18-20 "the summary frames may also be embedded in 
prerecorded media, such as, for example, video tape or video disks"). 

Regarding claims 27 and 57, Yeo et al suggest a system and method for 
processing video source frames comprising key frames in a first memory (Col 5, lines 1- 
2 "A fixed memory buffer in a computing device can be maintained to store the selected 
frames") and a mechanism to transfer the key frames from the first memory to the 
second memory (Col 2, lines 18-20 "the summary frames may also be embedded in 
prerecorded media, such as, for example, video tape or video disks"), but does not 
specifically disclose erasing the key frames after the transfer. 
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Dimitrova teaches deleting the key frames from the first memory after the 
transferring mechanism completes transfer of the key frames from the first memory to 
the second memory (Col 3, lines 31-32 "the data structure is transferred from the 
memory to the source tape." The examiner notes that the transfer of data or any other 
object implies the movement of that data or object from a source to a destination, and 
that the source no longer has the original data or object. If such a loss of the object 
from the source were intended, the term "copy" would be appropriate to the movement 
of data.) 

As suggested by Yeo et al and taught by Dimitrova et al, continuing to store the 
key frames in a first memory after their transfer to a second memory would be a waste 
of memory, and their deletion would free processor and memory for other tasks that 
may have more value to the viewer. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to include erasing the key frames from the first 
memory after transferring the key frames to a second memory. 

Regarding claims 28 and 58, Yeo et al disclose a system and method for 
processing video source frames wherein the video frame extraction algorithm comprises 
a content-based method of video frame extraction (Col 5, lines 15-16 "An alternative 
method to select the candidate summary frame is based on content-based sampling of 
video frames"). 

Regarding claims 29 and 59, Yeo et al disclose a system and method for 
processing video source frames with two methods (Col 4, line 66 - Col 5, line 1 "A 
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candidate summary frame is taken from the video signal at regular time interval t" and 
Col 5, lines 14-16 "An alternative method to select the candidate summary frame is 
based on content-based sampling of video frames"), but does not disclose four distinct 
methods of content-based keyframe extraction. 

Dimitrova et al teach a system and method for processing video source frames 
wherein the content-based method includes a keyframe scene detection method (Col 5, 
lines 27-40 describe "Method One"), a Method Two keyframe scene detection method 
(Col 6, lines 32-49 describe "Method Two"), a Method Three keyframe scene detection 
method (Col 7 lines 23-39 describe "Method Three"), and a Method Four keyframe 
scene detection method (Col 8, lines 19-45 describe "Method Four"). 

As suggested by Yeo et al and taught by Dimitrova et al, a variety of key frame 
extraction methods provides versatility and economy in storage and processing 
requirements, enhancing the user's experience. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to include four methods of key frame scene 
detection methods. 

Regarding claims 30 and 60, Yeo et al disclose a system and method for 
processing video source frames wherein the video frame extraction algorithm comprises 
a content-independent method of video frame extraction (Col 4, line 66 - Col 5, line 1 "A 
candidate summary frame is taken from the video signal at regular rime interval t"). 
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4. Claims 7 and 37 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Yeo et al and Dimitrova et al as applied to claims 1,6,31, and 36 above, and further in 
view of Martino et al (6,473,095). 

Regarding claims 7 and 37, Yeo et al suggest a special key frame with an 
indication that it is the last key frame (Col 5, lines 4-5 "In a live-broadcast, summary 
frames of past content can be selected." The most recent key frame is indicated by its 
position on the screen as shown in Figs 8A, 8B, and 8C). 

Martino et al teach a system and method for processing video source frames 
wherein the recording mechanism records the indication in a special keyframe that is 
appended to the extracted key frames in the first memory (Col 4, lines 21-22 "A program 
boundary is placed between H 7 and H 6 in accordance with box 209 [sic] of FIG. 2"). 

As suggested by Yeo et al and taught by Martino et al, indicating the final frame 
in a set of extracted key frames would provide the user with information regarding how 
recently the key frame review data can be accesses, further enhancing the use of the 
key frame extraction and display system to him or her. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify Yeo et al to include an indication of the last extracted key 
frame. 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to James A. Fletcher whose telephone number is (703) 
305-3464. The examiner can normally be reached on 7:45AM - 5:45PM M-Th, first 
Fridays off. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Faile can be reached at (703) 305-4380. 
Any response to this action should be mailed to: 



Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal 
Drive, Arlington, VA Sixth Floor (Receptionist). 

Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the Technology Center 2600 Customer Service Office 
whose telephone number is (703) 306-0377. 



Commissioner of Patents and Trademarks 



Washington, DC 20231 



or faxed to: 



(703) 872-9314 (for Technology Center 2600 only). 
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